Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TEZ-4542: Tez application may fail due to int overflow when record size is large and sort memory is low. #336

Merged
merged 1 commit into from
May 14, 2024
Merged

TEZ-4542: Tez application may fail due to int overflow when record size is large and sort memory is low. #336

merged 1 commit into from
May 14, 2024

Conversation

zhengchenyu
Copy link
Contributor

See TEZ-4542 for details.

@tez-yetus

This comment was marked as outdated.

Sign in to view
@zhengchenyu zhengchenyu deleted the TEZ-4542 branch February 23, 2024 08:52
@tez-yetus

This comment was marked as outdated.

Sign in to view
@zhengchenyu zhengchenyu reopened this Feb 23, 2024
@tez-yetus

This comment was marked as outdated.

Sign in to view
@tez-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 24m 13s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 16m 35s master passed
+1 💚 compile 0m 36s master passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu122.04
+1 💚 compile 0m 34s master passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚 checkstyle 1m 31s master passed
+1 💚 javadoc 0m 49s master passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu122.04
+1 💚 javadoc 0m 31s master passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+0 🆗 spotbugs 1m 42s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 1m 40s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 22s the patch passed
+1 💚 compile 0m 23s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu122.04
+1 💚 javac 0m 23s the patch passed
+1 💚 compile 0m 20s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚 javac 0m 20s the patch passed
+1 💚 checkstyle 0m 17s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 19s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu122.04
+1 💚 javadoc 0m 18s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~22.04-b08
+1 💚 findbugs 0m 57s the patch passed
_ Other Tests _
+1 💚 unit 9m 27s tez-runtime-library in the patch passed.
+1 💚 asflicense 0m 18s The patch does not generate ASF License warnings.
60m 10s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-336/4/artifact/out/Dockerfile
GITHUB PR #336
JIRA Issue TEZ-4542
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux e6c6a2d8c92f 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / f8c2e11
Default Java Private Build-1.8.0_392-8u392-ga-1~22.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~22.04-b08
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-336/4/testReport/
Max. process+thread count 2100 (vs. ulimit of 5500)
modules C: tez-runtime-library U: tez-runtime-library
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-336/4/console
versions git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@zhengchenyu
Copy link
Contributor Author

@abstractdog Can you please review this PR?

@abstractdog
Copy link
Contributor

abstractdog commented May 13, 2024 •
edited
Loading

the affected lines were touched by @rbalamohan in the past frequently, he is the PipelinedSorter expert, is there a chance you can take a look at this tiny patch?

@rbalamohan
Copy link
Contributor

LGTM. +1.

BTW, pipelinesorter is the brainchild of @t3rmin4t0r . :)

@abstractdog abstractdog merged commit 642f4c0 into apache:master May 14, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zhengchenyu @tez-yetus @abstractdog @rbalamohan